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A new  dual  optimization  framework  for  some  problems  of  information 
theory  and  statistics  is  developed  in  the  form  of  dual  convex  programming 
problems  and  their  duality  theory.  It  extends  the  work  of  Thames  and 
Cooper  for  finite  discrete  distributions  to  the  ease  of  general  measures. 
Although  the  primal  problem  (constrained  relative  entropy)  is  an  infinite  • 
dimensional  one,  the  dual  problem  is  a finite  dimensional  one  without 
constraints  anti  involving  only  exponential  and  linear  terms.  Applications 
range  from  mathematical  statistics  and  statistical  mechanics  to  traffic 
engineering,  marketing  and  economics. 


A DUAL  OPTIMIZATION  FRAMEWORK  FOR  SOME  PROBLEMS 
OF  INFORMATION  THEORY  AND  STATISTICS 


A.  Ben-Tal  and  A.  Charnes 


In  his  "Mathematical  Foundations  of  Statistical  Mechanics" 

Khinchln1  introduced  the  notion  of  conjugate  function  as  the  solution 
i , the  maximization  of  relative  entropy  subject  to  a single  constraint 
on  the  mean  of  the  distribution  sought.  In  their  developments  of 
information  theoretic  methods  in  mathematical  statistics,  Kullback  and 
Leibler2  made  a basis  for  treating  problems  of  statistical  estimation  and 

hypothesis  testing  which  was  extensively  developed  in  the  monograph  of 
Kullback  "Information  Theory  and  Statistics"3.  More  recently,  Akaike 
in  his  paper  "Information  Theory  and  an  Extension  of  the  Maximum  Likeli- 
hood Principle"  has  emphasized  the  great  breadth  and  depth  of  these 
Information  theoretic  methods  by  indicating  their  application  to  many 
classes  of  statistical  problems,  and  also  including  the  representation  of 
the  maximum  likelihood  principle  as  asymptot i c,  for  large  samples,  to  the 
decision  theoretic  approach  of  information  theory.  Again,  more  recent 
work  In  Irreversible  statistical  mechanics  by  B.O.  Koopman4  has  emphasized 
the  Importance  and  analytic  convenience  of  a const ra i ned  entropy  (or  In- 
formation) approach  in  deducing  Important  statistical  mechanics  phenomena 
with  a minimum  of  ad  hoc  hypothesis. 


In  all  of  this  work  the  ext reml zat ion  problem  has  been  solved 
explicitly  only  (as  in  Khinchln's  case)  for  a single  linear  equality 
constraint  in  non-negative  variables.  Not  until  the  work  of  Charnes  and 


Cooper5’6,  has  the  fact  been  brought  out  that  dual  convex  programming 
problems  are  Involved,  and  that  the  dual  of  the  constrained  entropy  problem 
is  In  terms  of  exponential  and  linear  functions  In  unconstrained  variables. 
The  work  of  Charnes  and  Cooper,  while  tying  In  the  method  to  other  problems 
of  traffic  engineering  and  economics,  as  well  as  providing  a complete 
characterization  of  the  duality  states,  has  encompassed  explicitly  only 
the  case  of  finite  discrete  distributions  (measures). 

It  is  the  purpose  of  this  paper  to  extend  these  results  and  develop 
a dual  optimization  framework  that  can  adequately  handle  these  classes  of 
problems  of  information  theory  and  statistics.  In  particular  we  have 
developed  a complete  duality  theory  for  the  case  of  general  as  well  as 
finite  measures. 

Although  our  primal  problem  Is  an  infinite  dimensional  one  (with  fini- 
tely many  constraints)  the  dual  problem  is  a finite  dimensional  one,  without 
constraint  and  involving  only  exponential  and  linear  terms.  As  we  show 
elsewhere,  such  a dual  optimization  framework  with  convenient  analytical 
functions  In  an  unconstrained  dual  seems  to  be  a unique  property  of  the 
information  theoretic  functional. 

The  paper  is  conveniently  summarized  by  the  titles  of  its  sections 


os 

fol lows : 

1. 

A formal  statement 

of  the  problem. 

2. 

Some  pre) imlnar les 

from  Convex  Analysis. 

3. 

Linearly  constrained  convex  programs  and  their  duals. 

it. 

Conjugates  and  subgradient  of  integral  functionals. 

5. 

A comp lete  dua lily 

theory  for  problem  (A). 

6. 

The  case  of  probabi 

1 i ty  measures . 

7. 

General  I zat i ons . 

t 
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l.  A FORMAL  STATEMENT  OF  THE  PROBLEM 

Let  T be  an  arbitrary  set,  F the  o-fleld  of  Bore!  subsets  of  T, 
dt  a non-negative  regular  Bore!  measure  (rBm)  on  T and  M(T)  the  linear 
space  of  real-valued  finite  rBm's  on  T.  For  an  element  p € M(T)  we  shall 


denote  by  jjr  Its  Radon -Nikodym  derivative. 


For  a given  summabie  positive  function  c:  T -*■  R;  continuous  functions 

». 

F|  : T -*■  R (i  ■ 1 . . ,m)  and  real  scalars  6 { ( i • 1 . . ,m)  we  seek  to  solve 

the  following  problem 


inf  | u(t)log[  “IyJ*  3dt 


subject  to 


(1)  [ u(t)F. (t)dt  - 6.  i * I m. 


(2)  u - & , p£  M(T) 

(3)  P non-negat I ve  and  absolutely  continuous 

(wi th  respect  to  dt) . 


Consider  the  linear  operator  A:  M(T)  -*■  Rm  given  by 


U — > 


Ff  (t)  dw 


l 'jo** 


and  the  integral  functional 


V 


L 


- i* 


l(u)  * 


UlO'os^U. 


if  m is  an  absolutely 
continuous  non-negative  rBm 

and  u ■ . 


otherwise. 


Then  Problem  (A)  can  be  written  as 
(P)  lnf{J(u):  Ap  ■ 6 , n 6 S) 

where  6 - (8, 8 and  S - domJ  -(u  : J(u)«“} 

1 ID 

It  will  be  shown  later  (Section  5)  that  J is  a convex  functional,  and 
so  (P)  Isa  linearly  constrained,  convex  optimisation  pt'oblem.  In  Section  3 
we  study  such  programs  and  Introduce  a duality  theory  for  them.  Before 
doing  so  we  collect  in  the  next  section  certain  material  from  Convex 
Analysis  needed  in  the  sequel. 

Throughout  the  paper  we  assume  that  the  linear  system  Ap  ■ e is 
irreducible  , i.e. 

Range  A ■ Rm. 

In  the  finite  dimensional  case  (A  is  an  m*n  matrix)  this  assumption  means 
that  A Is  of  full  row  rank  so  that  none  of  its  m equations  a'w  ■ 6( 

(I  ■ l,...,m)  (A*  the  i-th  row  of  A)  Is  redundant.  Hence  the  termin- 

ology "Irreducible". 


2.  SOME  PRELIMINARIES  FROM  CONVEX  ANALYSIS 

vector 

Let  E and  E be  real /spaces,  and  <,>  a bilinear  function 

. * A A A 

defined  on  pairs  (x,x),x€E,x  6 E • Let  E and  E be  equipped 

with  locally  convex  Hausdorff  topologies,  compatible  with  the  bilinear 

form,  so  that  every  element  of  one  space  can  be  identified  with  a con- 

a 

tinuous  linear  functional  on  the  other.  In  this  case  E and  E are 
called  paired  epaoee  and  <,>  is  the  pairing.  (For  more  information 
see  I 7,  Chapter  IV]. 

A function  f:  E -+•  R is  oonvex  if  for  every  x^  ,Xj  € E and  0 < A < I 

f (Ax  + ( 1 - A) y ) < Af (x)  + (l-A)f(y), 

f is  proper  if  it  is  bounded  below  and  it  is  not  Identically  +®. 

I f , for  all  x , 

f (x)  ■ I lm  inf  f (z) 
z*-x 

then  f is  louer  semi -continuous  (l.s.c). 

•k  k 

The  function  f : E ■+  R given  by 

f*(x*)  - sup{<x,x*>  - f(x)) 
x 

Is  called  the  (convex)  conjugate  of  f.  This  is  always  a l.s.c.  convex 
function.  Conversely  if  f is  a l.s.c.  proper  convex  function  then 

f (x)  - suj>{<x,x*>  - fV)>, 

X 

I .e. 


I 


A vector  x*  € E*  is  a ait  gradient  of  a convex  function  f at  x if 
(k)  f(z)  > f(x)  + <z-x,x*>  for  all  z 6 E. 

The  set  of  all  subgradients  of  f at  x is  denoted  by  3f(x).  If  f is 

finite  and  differentiable  at  x,  with  f‘(x)  denoting  Its  /Frechet)  derivative, 
then , 

3f(x)  - { f(x)>. 

A function  g Is  oonoave  If  -g  is  convex.  The  (concave)  conjugate 

a 

of  such/ funct Ion  is  defined  by 

g*(y*)  - • n f { <y  * ,y>  - g(y)h 

y 


3.  LINEARLY  CONSTRAINED  CONVEX  PROGRAMS  AND  THEIR  DUALS 

'■  | 

Let  E and  F be  real  vector  spaces,  A:  E -*•  F a linear  operator, 

. 

h:  E ■*  R a convex  function  with  dom  h ■ S and  g:  F -*■  R a concave 
function  with  dom  g ■ 8. 

Consider  the  primal  problem 

(I)  lnf{h(x)  - g(Ax) : x € S,  Ax  C B). 

The  Fanohel -Rookafellar  duality  theory  [8  ] associates  with  (l) 
the  dual  problem 

(II)  sup(g  (x  ) - h (A  x ) : x €B,Ax  €S) 

a h * a a 

where  A : F E is  the  adjoint  of  A,  E and  F are  t'<e  spaces  paired 

] 

. . ... 
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I 


with  E and  R (with  the  pairing  respectively  and  h* , 

* / \ 

g are  the  convex  (resp.  concave)  conjugates  of  h and  g,  i.e. 

h (•)  ■ sup(<x,*>_  - h(x)  : x € S) 
g*(*)  - inf(<y ,*>p  - g(y) : y € B). 

The  main  result  concerning  (I)  and  (II)  is  that  if  the  auper- 
oonaiatenoy  aaaurnption  holds,  I.e. 

(5)  3 x 6 S such  that  Ax  6 int  B 

then  inf(I)  ■ max(n).  Dually  if 

,ri,  * * A * >* 

( •’  ) 3 x € S such  that  A x 6 int  B 

Then 

min  (I)  ■ sup(n)  . 

Furthermore,  whenever  min(I)  - max(n)  , a pair  x,  x solves  (l)  and  (U) 
respectively  if  and  only  If  (see  [C],  p.l8S). 

■ - * * * - * A 

( *>)  x C 3h  (A  x ) , Ax  € 3g  (x  ) . 

It  should  be  mentioned  that  in  the  absence  of  assumption  ( 5)  , (5‘) 
or  similar  assumption,  one  still  has  the  so  called  weak  duality  relation: 
inf  (I)  > sup(  □) . 


We  write  'min'  ('max')  If  the  Infimum  (supremum)  is  attained. 
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Here  we  consider  the  following  special  case  of  (I): 

(P)  lnf(h(x):  Ax  • b,  x € S) 

and  we  further  assume: 

(i)  E is  a Banach  space,  F Is  a Hilbert  space  (with  inner  product  <,>) , 

(i i)  int  S 4 9 ’ 

(ill)  [“ i rreduci bi I i ty  assumption")  Range  A - F. 

flote  that  (P)  corresponds  to  (I)  with  B ■ { b ) and 
g(')  - <$(*|8)  « the  Indicator  function  of  B. 


This  implies: 

ft  . ft.  ft 

g (x  ) - <b,x  > 

therefore,  the  dual  of  (P) 
(0)  supi<b,x  > - h 


* ft  * ft 
(Ax):  Ax 


€ S* ) . 


Unfortunately,  the  superconsistency  assumption  (5)  does  not  hold 
here  since  Int  B - 0.  However,  we  shall  make  use  of  a less  familiar 
regularity  condition  ((9).  P-50),  which  for  the  pair  (P)-(D)  reduces  to 

(7)  0 € core(A(S)  - b) . 


This  condition  also  implies  that  Inf(P)  ■ max(D).  We  recall  that  for  a 
subset  Q c F 

core  Q - (q  fc  Q:  Vv  £ F,  3e  > 0 such  that  q ♦ Xv  € Q for  all  X€(-e,tJ). 
If  Q is  a convex  set  with  nonempty  Interior 


core  Q ■ int  Q. 


The  following  lemma  shows  that  the  I rreducibi I i ty  assumption  Is 
essential  for  the  validity  of  (7). 


Lemma  I.  Regularity  condition  (7)  holds  only  If  assumption  (ill)  holds. 


Proof.  Suppose  that  (7)  holds,  then 
(a)  3x  £ S 3 Ax  - b 


(b)  Vv  £ F,  3c  > 0 such  that,  for  every  A£[-e,e],  3x  £ S satisfying 
Ax  ♦ Av  • b. 

Since  F is  a Hilbert  space 


(8) 

F ■ Range  A 

♦ N(A  ) , 

whe  re 

N denotes  nul 1 

space , 

thus  if  (iii)  does  not  hold 

- * . 

(9) 

3v  £ N ( A ) , 

v ¥ 0. 

At  the 

same  time  , (8) 

implies 

, by  (a)  , that 

(10) 

b i.  N(A*). 

Le  t x 

be  a solution  i 

of  Ax  ♦ 

Av  “ b for  some  given  A > 0 

exists 

by  (b)).  Now 

a-  - 


0<  A "-v.v*  ■ «'V,b-Ax>  • 'V,b>*<A  v,x>  ■ 0 by  (9)  and  (10). 

This  contradiction  shows  that  (III)  must  hold  whenever  (7)  Is  valid. 


□ 


The  regularity  condition  (7)  Is  not  easy  to  check,  therefore  we 
introduce  in  the  following  lemma  a much  simpler  one. 


1 

J 


10  - 


Lemma  2.  If  (P)  satisfies  (l)  (II)  and  (III),  then  condition  (7)  Is 
Implied  by  the  following  atriot  feaa  ibility  assumption: 


(11)  3x  € IntS  satisfying  Ax  ■ b. 


Proof.  Let  x satisfy  (11).  Note  that,  since  S Is  convex  and 
IntS  9 9,  IntS  - coreS.  Now  x € cores  If  and  only  If 


x + Ax  £ S fo  r all  x and  all  A € (_c  ej  for  some  e > 0. 


In  particular 


x ■ x ♦ Ax(v)  £ S 


where  x(v)  is  a solution  of 


Ax  ■ v. 


VA  £ [-e  ,t ] 


(That  such  a solution  exists  follows  from  the  I rreduclbl 1 1 ty  assumption.) 
Further 


b-Ax  ■ b - A(x  + Ax(v))  ■ b - Ax  + AAx(v)  ■ u + Av. 


The  latter  shows  that  for  every  v € F,  there  exist  x satisfying 


Ax  + Av  ■ b A £ (-e,e)  for  some  e > 0, 


0 £ core (A(S) -b) . 


We  will  summarize  the  results  concerning  the  linearly  constrained 
problem  (P)  and  Its  dual  (D)  in  the  following 


Theorem  I Consider  problem  (P)  and  assume  that  (I),  (II)  and  (III) 


(2)  Whenever  min  (P)  - max  ( D) , a necessary  and  sufficient  conditions 

for  a pair  xQ  € S,  x*  € S*  to  solve  (P)  and  (D) , respectively,  are 

Ax  ■ b and 
o 

(12)  x € 3h  (A  x)  . 

o o 

Proof.  Condition  (il)  Implies  (7),  by  Lemma  2,  and  the  latter  Implies 

Inf (P)  ■ max(D)  by  the  above  cited  result  [9  , p.  50].  The  optimal  tty 

conditions  (12)  are  the  specialization  of  (6)  to  our  special  case. 

Indeed,  for  g(*)  ■ 6 ( • | { b > ) we  have  g*(*)  ■ <b,*>  so  that 

3g  (x*)  - (b)  . 

□ 


k.  CONJUGATES  AND  SUBGRADIENTS  OF  INTEGRAL  FUNCTIONALS 

in 

The  duality  theory  presented  In  the  previous  section  Is  stated  /terms 
of  the  conjugate  function  of  the  objective  function,  and  Its  subgradient. 

In  Problem  (A)  however,  the  objective  function  Is  the  integral  funational 
J(u).  Therefore,  we  shall  collect  In  this  section  results  concerning  the 
computation  of  J*  and  3J(«).  For  more  details  the  reader  Is  referred 
to  [ 10  ] and  [ 11]  . 

Let  C(T)  be  the  vector  space  of  continuous  functions  x:  T R with 
the  norm 

INI  - max{ | x ( t ) | : t € T). 


We  recall  that  the  space  M(T)  of  Section  I is  the  dual  of  C(T).  Further 
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let  f:  T x R -*>  R be  a function  satisfying 

(a)  Vt  € T,  f(t,*)  is  I.s.c.  proper  convex  function. 

(b)  f(t,x)  is  measurable  in  t for  all  x € R and  the  (nonempty  convex) 


D(t)  • (x  £ R:  f (t ,x)  < • ) 
has  a nonempty  interior. 

(c)  D(t)  Is  fully  l.B.o.  (see  [10, p. 457)  this  condition  holds  e.g.  when 
D,(t)  does  not  depend  on  t.  The  latter  is  enough  for  our  purposes). 

Define  the  integral  functional  on  C(T) : 

If(x)  - J f(t,x)dt. 

The  conjugate  of  If  is  then,  by  definition 

I*(u)  - sup  { [ xdp  - [ f(t,x(t))dt:  x € C(T)). 

JT  JT 


The  following  result  follows  directly  from  [10  J.  We  remark  that 
conditions  (a)  and  (b)  above  imply  that  f(t,x)  is,  so-called,  normal 
integrant . 

Lemma  3 If  f(t,x)  is  a summabte  function,  for  every  x € R,  and 

satisfies  (a),  (b)  and  (c)  then 

(A)  Is  well-defined,  finite,  continuous  and  convex  function  on  C(T) ; 

(B)  the  conjugate  of  If  is  the  function  I*:  M(T)  R given  by: 


l*(v)  - 


[f  (t,  & ) if  p is  absolutely  continuous  with  respect 
J dt  to  dt 


otherwise. 


where  f*(t,x*)  Is  the  conjugate  of  f(t,*)  evaluated  at  x*.  □ 


- »3  - 


Next,  one  can  derive  from  [10,  Cor.  4B],  the  following  formula 
for  computing  the  subgradient  of  1^. 

Lemma  4 Under  the  assumption  of  Lemma  3,  and  the  following  additional 
assumption 

(d)  (f(t,u(t)  + x)  is  summable  for  all  x in  some  neighbourhood 

of  zero  # 

It  follows  that  li  € 3I^(u)  If  and  only  If  almost  everywhere  (a.e.) 

(14)  ^ € 3f (t  ,u(t) ) . almost  everywhere  (a.e) 

Here  3f(t,u)  is  the  subgradient  of  f ( t , * ) at  u.  □ 


5.  A COMPLETE  DUALITY  THEORY  FOR  PROBLEM  (A) 


We  return  to  the  setting  described  in  Section  I. 

Consider  the  integral  functional  I:  C(T)  ■*  R, 

I(x)  - |TC(t)ex(t)'1dt  . 

The  integrand  f(t,x)  - c(t)X^  1 clearly  satisfies  assumptions  (a),  (b) 
and  (d)  of  Section  4.  (Recall  that  c(t)  is  summable  and  positive.) 

The  conjugate  of  f(t,*)  I s by  definition 

. 

f*(t,x*)  • sup  (xx*  - c(t)ex  . 
x€R 

The  sup  can  be  easily  computed  by  equating  the  derivative  of  the  supremand 
to  zero,  so  one  obtains 


i 


(we  use  the  convention  Olog  0*0). 

It  follows  from  Lemma  3 that 

(A)  I(x)  is  a continuous  convex  functional 

(B) 

I*(u)  - J(m) 

where  J(u)  is  the  integral  functional  defined  in  Sect  ion  (I). 

The  last  relation  immediately  implies  (see  Section  2)  that  J is  a 
l.s.c.  convex  functional.  Moreover,  in  regard  of  the  continuity  of  I 

( 15)  J (u)  • I**(u)  ■ I (u) 

( 16)  dom  J*  - C(T) . 

We  further  note  that  the  adjoint  of  the  linear  transformation  A of 
Section  1 is  the  mapping  A : Rm  -*-C(T) 

(x. . . »x  ) -*  £ x.F.  (t) . 

1 m i • 

We  have  now  all  the  elements  needed  to  specify  the  dual  problem 

of  (A),  as  described  In  Section  3.  In  fact,  problem  (D)  becomes  here 

m * 

2 x.F.(t)  - 1 
m . r i-1 

(6)  sup  { £ x{ 0 { - c(t)e  dt), 

i»1 

an  unoonetrained  finite  dimensional  concave  program.  (Thn  r-.-lation 
A x € S is  here  x*€  Rm  since,  by  (16),  S*  - C(T) , i.e.  the  whole  space.) 


MPllllll  , rnwmm,,  RM* . 
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The  relations  between  the  primal  problem  (A)  and  its  dual  ( B ) are 
expressed  in  the  following  results. 

Theorem  2 The  supremum  of  Problem  (B)  is  attained  only  if 


(17) 


f 3 positive  (a.e.)  rBm  p,  whose  derivative  u - 
satisfies  the  linear  equations  (1). 


Proof.  Since  (B)  is  an  unconstrained  problem,  its  supremum  is 
attained  (say  at  x*)  only  if  x*  is  a critical  point  of  the  supremand. 

a A 

I ,e . x i s a solut i on  of 


l.  - [ c(t)F.  (t)e 

1 h ' 


Si  F.(t)-1 


dt  * 0 i * I , . . . ,m 


which,  by  the  definition  of  the  mapping  A,  is  nothing  else  but 


Ap  * 6 


where  p is  the  measure  with 


)V  . . 

Sx.F.(t)-1 


“ " 3t  " c(t)e 


Hence  (17)  is  satisfied  by  U ■ P 


□ 


Theorem  3.  Problem  (B)  is  bounded  above  If  and  only  if  Problem  (A) 
has  a feasible  solution. 


Proof.  The  "if"  part  follows  from  the  weak  duality  relation 
inf(A)  >sup(B).  We  proceed  to  prove  the  "only  If"  part. 


Suppose  that  (A)  has  no  feasible  solution  and  consider  the  following 
subset  of  Rm 

K - (y  € Rm:  y - Ay  for  some  y € M(T)  satisfying  (3)  } . 

By  our  assumption  then 

(19)  0 € K. 

there 

Since  K Is  a closed  convex  cone,  it  follows  from  (19)  that/exists 

whieli 

a hyperplane,  passing  through  the  origin,  / strictly  separates  6 
and  K.  I.e. 

f z'y  < 0 Vy  € K 
3 z £ R , z ¥ 0 such  that  s 

lz'6>0 

or 

t z'(Ay)  <0  Vy  € M(T)  satisfying  (3) 
l z'0  >0 

or 

| <A  z,y>  <0  Vy  € M(T)  satisfying  (3) 

*•  z'e  > o 

Now 

* f * 

<A  z ,y>  ■ ]^.(A  z)dy  and  the  latter  is  non-positive  for  every 
non-negative  rBm  only  if 

A*z  < 0 (a. e) . 

We  conclude  that 

(20)  30  z € Rm  such  that  A z < 0 (a.e.)  and  z'0  > 0. 


Note  that  Problem  (B)  is  in  fact 


Let  z be  the  vector  in  (20),  then,  with 


x ■ Mz  (M  positive  scalar), 


the  objective  function  in  (21)  can  be  made  arbitrary  large  by  choosing 
M large  enough.  □ 


Theorem  k If  Problem  (A)  is  feasible  then  the  infinimum  is  attained  and 


(22)  min (A)  • sup(B) . 


If  Problem  (A)  is  strictly  feasible,  i.e.  (17)  Is  satisfied^  then 


(23)  min (A)  ■ max(B) . 


Proof.  Since  for  Problem  (A):  S • C(T)  and  B ■ Rm  it  follows 
that  condition  (51)  trivially  hold*  and  hence  the  conclusion  (22). 

Now,  for  Problem  (A),  the  strict  feasibility  assumption  (II)  reduces  to 
(17)  and  so  (23)  follows  from  the  first  conclusion  in  Theorem  I and  (22). □ 

The  last  result  gives  the  optimality  conditions  for  the  pair  of  dual 
Problems  (A)  and  (B). 

Theorem  5.  Let  (A)  be  strictly  feasible.  Then  u € M(T)  and  x*  € Rm 
are  optimal  solution  of  (A)  and  (B) , respectively,  If  and  only  if  u 
satisfies  (1),  (2),  (3)  and 


Proof.  The  optimality  conditions  expressed  in  the  second  part  of 
Theorem  1 are  here:  (i)  feasibility  of  p and  (ii) 


(25)  P e 3J*(Ax*  ) . 

Now,  by  (15),  J (u)  ■ I(u)  • I c(t)eU^  1 dt  so  that,  by  Lemma  (4), 

JT 

(25)  hold*  if  and  only  if 

(26)  € 3f(t,Ax*)  (a.e. ) 

where  f(t,u)  » c(t)eu  ' whose  subgradient  simply  coincideswith  its 
derivative.  Hence  (26)  is  equivalent  to 

- 

du  A x *1  / v 

- c(t)e  (a.e.) 

which  is  just  (24) . □ 


6.  THE  CASE  OF  PROBABILITY  MEASURES 

In  this  section  we  study  problems  of  type  (A)  with  the  additional 
constraint 


in  which  case  u becomes  a probability  measure.  Of  course  one  can  write 


and  derive  the  following  dual  problem: 
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m * ft 

sup  { 2 x. 0.  + x 
1-1  1 1 m 


* Cl  ' [ci 


* . . * 
,f1xiFi(t)  + vr1 


an  unconstrained  problem  In  R . However,  one  can  derive  the 
fol lowing  dual 


sup  { Ix.6.  - log 


fTC(t) 


2x?F.(t) 
e 1 ' dt). 


an  urloons  trained  problem  in  iF.  To  derive  (B ' ) from  (28),  one  merely 
maximizes  the  objective  function  In  (28)  with  respect  to  x^j  for 

fixed  (X| xffl) , which  can  be  done  by  equating  Its  derivative  to  zero. 

The  analytic  solution  thus  obtained 


x—i  * -,09 


[ c ( t) 
JT 


Zx,F.(t)-1 

e dt 


is  then  substituted  again  in  (28)  and  the  result  is  indeed  (B1) 


All  the  result  of  the  previous  section  can  be  applied  to  the 
present  case.  If,  In  condition  (17),  we  add  the  requirement  that  u 
satisfies  (27) , then  Theorems  2,3,  and  4 remain  valid.  In  Theorem  5 
the  optimality  condition  (24)  has  to  be  replaced  by 


du  c(t)e 
dt 


(t) 


Zx.F|(t) 

c(t)e  dt 


(a.e. ) 


f 
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7.  GENERALIZATIONS 

The  reason  that  the  dual  Problem  (B)  is  unconstrained  is  that 
$*  $ dom  J*  - C(T) . 

This  in  turn  holds  since 

dom  f ( t . • ) - R ; 

which' in  turn  results  from  the  fact  that 

(29)  tenge  gj  f (t,x)  - R. 

Here  f was  the  Integrand  f(x,t)  ■ x log  • 

These  observations  lead  to  the  following: 

Proposition  Consider  a primal  problem  of  the  form 

(30)  Inf  { Jf(t,g£)dt:  w satisfies  (1 ),  (2) , and  (3) ) 

And  assume  that  f Is  a norma)  convex  Integrand,  differentiable  In  x 
for  all  t € T and  satisfying  (29).  Then  Problem  (30)  has  the  following 
unconstrained  dual : 

(31)  sup  (e'x*  - f f*(t,  Ix?F,(t))dt)  . 

x*€Rm  h 1-1  1 ' 

f*(t,y)  • yr(t,x*)  - f(t,r(t,x*)) 


where 


I 

I 


and  where  x • r(t,x  ) is  a solution  of  the  equation 

g-  f(t,x)  - x*  . □ 

If  f(x.t)  satisfies  condition  (a)-(d)  of  Section  (k)  , then  results 
analogous  to  Theorems  k and  5 are  valid  for  the  dual  pair  (30) -(30  • 

It  Is  also  easy  to  see  from  the  proof  of  Theorem  3 that,  if 

V lim  f*(t,y)  < - , 

then  Problem  (31)  is  bounded  above  if  and  only  if,  Problem  (30)  has  a 
feasible  solution. 

We  finally  remark  that  the  constraints  (I),  can  be  replaced  by 
the  more  general  constraint 

|^u(t)K(t,s)dt  • b(s) , 

where  b .is  an  element  of  an  Hilbert  space.  The  generalization  of 
Theorems  2-5  to  this  case  is  straightforward. 


I 
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